Papers with long-term planning
BioPlanner: Automatic Evaluation of LLMs on Protocol Planning in Biology (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have impressive capabilities on a wide range of tasks, such as question answering and the generation of coherent text and code. |
| Approach: | They propose a framework for automatic evaluation of large language models on open-ended planning problems and a dataset of biology protocols with corresponding pseudocode representations. |
| Outcome: | The proposed framework evaluates an LLM on a dataset of biology protocols with corresponding pseudocode representations. |
ProcWorld: Benchmarking Large Model Planning in Reachability-Constrained Environments (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for embodied spatial reasoning and long-term planning are non-trivial due to the combinatorial complexity of long-horizon abstract reasoning. |
| Approach: | They propose a large-scale benchmark for partially observable embodied spatial reasoning and long-term planning with large language models and vision language models. |
| Outcome: | The proposed model performs better in 16 task types, 5,000 rooms, and over 10 million evaluation trajectories with diverse data distribution. |